Skip to content

[Bugfix] Fix depyf incompatibility with Inductor's set_sys_modules - #60873

Merged
AndreasKaratzas merged 2 commits into
vllm-project:mainfrom
stefankoncarevic:fix-depyf-set-sys-modules
Oct 30, 2026
Merged

AndreasKaratzas merged 2 commits into
vllm-project:mainfrom
stefankoncarevic:fix-depyf-set-sys-modules

Conversation

@stefankoncarevic

@stefankoncarevic stefankoncarevic commented Oct 9, 2026 •

Copy link
Copy Markdown
Collaborator

Overview

Any torch.compile run with debug_dump_path set fails once compilation reaches an autotune code block, because depyf's load_by_key_path hook does not accept Inductor's set_sys_modules argument. A compatibility shim gives the hook **kwargs and forwards them.

Seen on the PyTorch Fullgraph step in Buildkite #93716.

Claims

  • Fixes InductorError: RuntimeError: Failed to run autotuning code block: patched_load_by_key_path() got an unexpected keyword argument 'set_sys_modules'.
  • Not platform-specific. The failing step is a ROCm mirror, but nothing in the cause or the fix is vendor-dependent; it needs only a cold compile cache.
  • The shim is conditional on depyf's hook signature, so it disables itself if depyf ever ships a fix.
  • The existing depyf test could not catch this. It was one parametrization of test_custom_compile_config, which reuses vLLM's on-disk compile cache, so on a persistent agent it usually verified nothing. It is now a separate test that disables the cache.
  • Adds a CPU-only unit test that reproduces the incompatibility in 2.3 s, instead of only through a GPU end-to-end run.
  • Costs 27 s on PyTorch Fullgraph and 11 s on PyTorch Compilation, both against budgets of 90 and 30 minutes. The other six parametrizations keep using the cache.
  • Not fixed: depyf's other two hooks are unchanged. Both were audited and are already compatible.

Validation

1× AMD MI355X (gfx950), ROCm 7.2, PyTorch 2.13.0+git733fca1, depyf 0.20.0, vLLM 19c0bed584.

Check Result
New unit tests, CPU only 6 passed, 2.3 s
End-to-end depyf test, cold cache, after the fix passed, 50.6 s
Same, before the fix failed, 25.8 s
Same, before the fix, warm cache passed, 23.3 s
Unit tests with the shim forcibly disabled 4 of 6 failed

The last two rows are the ones that matter. The warm pass is why this reached main: with a populated cache nothing is compiled, depyf's hook is never called, and the test is green. The forced-disable run confirms the new tests measure the fix rather than the environment.

Details

depyf.prepare_debug replaces PyCodeCache.load_by_key_path with a hook that dumps each generated kernel before loading it. The hook's signature is frozen at (key, path, linemap, attrs), and generate_and_run_autotune_block now also passes set_sys_modules. The resulting TypeError is raised inside the exec of the tuning block, so it surfaces as an InductorError about the autotune block rather than as anything mentioning depyf.

Bumping the pin is not available: 0.20.0 is the last release upstream published, its repository has had no commits since that release, and four pull requests have been open without review. The hook is therefore patched in vLLM, in the way depyf already handles this for lazy_format_graph_code: accept **kwargs and forward them.

The shim forwards the keyword rather than dropping it. Dropping it would also make the call succeed, while quietly reintroducing the sys.modules exhaustion that pytorch#184285 fixed; one of the unit tests pins exactly that. Both vLLM entry points into depyf go through the shim, so neither can miss the patch.

Pull Request Checklist
  • I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

  • AI assistance was used during the creation of this PR.

  • Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

  • Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

  • Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

  • Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

depyf's load_by_key_path hook has its signature frozen at
(key, path, linemap, attrs), while Inductor also passes set_sys_modules
since pytorch#184285. Every compilation that reaches an autotune code
block with debug_dump_path set therefore fails. depyf 0.20.0 is the last
release upstream published, so patch the hook in vLLM instead: give it
**kwargs and forward them, guarded by a signature check so it disables
itself if depyf is ever fixed.

The existing depyf test reused vLLM's compile cache and so usually
verified nothing. Split it out with the cache disabled, and add a
CPU-only unit test for the patch itself.

Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@stefankoncarevic stefankoncarevic added the ready ONLY add when PR is ready to merge/full CI is needed label Oct 9, 2026
@github-actions

github-actions Bot commented Oct 9, 2026

Copy link
Copy Markdown

✅ @stefankoncarevic, CI is now available for this PR.

  • /ci run starts upstream CI; /amd-ci run starts AMD CI only.
  • Your branch must contain every commit currently on its upstream target branch. Merge or rebase onto the latest target branch, then rerun the command. Append --allow-stale to a run command to test an outdated branch at your own risk.
  • /ci retry retries failed jobs in the CI build for the current PR head. If the current head has no CI build, it starts a new CI build for the current head containing only jobs that failed in the latest earlier CI build for this PR.
  • /amd-ci retry retries failed jobs in AMD CI for the current PR head. Use /amd-ci run when the current head has no AMD CI build.
  • /ci cancel cancels scheduled or running CI builds for this PR branch; /amd-ci cancel does the same for AMD CI only.

@stefankoncarevic

Copy link
Copy Markdown
Collaborator Author

/ci run

@github-actions

github-actions Bot commented Oct 9, 2026

Copy link
Copy Markdown

✅ Triggered Buildkite CI #93870 for commit 9cf51b547847.

@yzong-rh

yzong-rh commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

LGTM, thanks! Should resolve the compile/fullgraph/test_full_graph.py::test_custom_compile_config[compilation_config3-facebook/opt-125m-model_kwargs3] failures in CI. E.g. main-branch Buildkite run #98732 that were exposed by #64635.

A more definitive fix should belong in depyf, but this should unblock

@AndreasKaratzas

Copy link
Copy Markdown
Member

Merging to unblock CI.

@AndreasKaratzas
AndreasKaratzas merged commit 2fab358 into vllm-project:main Oct 30, 2026
124 checks passed
@github-project-automation github-project-automation Bot moved this from To triage to Done in torch.compile integration Oct 30, 2026
ChengYM5685 added a commit to ChengYM5685/vllm that referenced this pull request Oct 31, 2026
Includes the set_sys_modules compatibility fix from vllm-project#60873 and resolves the
ModelConfig type import conflict while preserving the MoRI EPv2 changes.

Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: ChengYM <320424417+ChengYM5685@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ready ONLY add when PR is ready to merge/full CI is needed torch.compile

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants